What Is Coreference, And What Should Coreference Annotation Be?

In this paper, it is argued that 'coreference an-notation', as currently performed in the MUC community, goes well beyond annotation of the relation of coreference as it is commonly understood. As a result, it is not always clear what semantic relation these annotations are actually encoding. The paper discusses a number of interrelated problems with coreference annotation and concludes that rethinking of the coreference task is needed before the task can be expanded (e.g., to cover part/whole relations) as has recently been advocated. As a step towards solution of the problems with coreference annotation , one possible simplification of the annotation task is suggested. This strategy can be summed up by the phrase "Coreference annotation should annotate coreference relations, and coreference relations only". 1 Introduction: Coreference Annotation Various practical tasks requiring language technology including, for example, information extraction and text summarization, can be done more reliably if it is possible to automatically find parts of the text containing information about a given topic. For example, if a text sum-marizer has to select the most important information , in a given text, about the 1984 Wall Street crash, then the summarization task is greatly helped if a program can automatically spot all the clauses in the text that contain information about this crash. To 'train' a program of this kind, extensive language corpora have been prepared in which human readers have annotated what has been called the coref-erence relation. These annotated corpora are then used as a 'gold standard' against which the program's achievements can be compared.